On the almost sure central limit theorem for ARX processes in adaptive tracking

The goal of this paper is to highlight the almost sure central limit theorem for martingales to the control community and to show the usefulness of this result for the system identification of controllable ARX(p,q) process in adaptive tracking. We also provide strongly consistent estimators of the even moments of the driven noise of a controllable ARX(p,q) process as well as quadratic strong laws for the average costs and estimation errors sequences. Our theoretical results are illustrated by numerical experiments

PI controller tuning for load disturbance rejection using constrained optimization

© 2016, Springer-Verlag Berlin Heidelberg. In this paper, a simple and effective PI controller tuning method is presented. To take both performance requirements and robustness issues into consideration, the design technique is based on optimization of load disturbance rejection with a constraint either on the gain margin or phase margin. In addition, a simplified form of the resulting tuning formulae is obtained for first order plus dead time models. To demonstrate the ability of the proposed tuning technique in dealing with a wide range of plants, simulation results for several examples, including integrating, non-minimum phase and long dead time models, are provided

Differential observation and integral action in LTI state-space controllers and the PID special case

This paper makes the case that practical differentiation of measured state variables may be seen as an observation or estimation scheme for linear time invariant state space controllers. It is shown that, although not having the separation property, the estimation error of this scheme converges to zero if the resulting closed loop system is strictly stable. On the basis of this concept, it is shown that PID controllers may be interpreted as a special case of state space controllers endowed with differential observation. The interesting consequences of this interpretation are discussed.Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/202

Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

A novel coupling control with decision-maker and PID controller for minimizing heating energy consumption and ensuring indoor environmental quality

Due to climate change, global energy crisis, and high-quality life requirement for people, decreasing building energy consumption and enhancing indoor environment quality through control of heating, ventilation, and air conditioning systems tend to be increasingly important. Therefore, favorable control methods for heating and ventilation systems are urgently necessary. In this work, a new coupling control with decision-maker was proposed, developed, and investigated; meanwhile, several demand controlled ventilation strategies combined with heating control method was compared considering heating energy consumption, thermal comfort, and indoor air quality. In order to properly model the service systems, the air change rates and thermal time constants have been first measured in a reference office installed with commonly applied bottom-hinged tilted windows in our low-energy building supplied by geothermal district heating. Then, simulations have been carried out across two typical winter days in the reference office. The results illustrate that the proposed combination of suitable heating and demand controlled ventilation coupling control methods with decision-maker and proportional-integral-derivative (PID) controller could greatly reduce heating consumption in the reference room during the office time: around 52.4% (4.4 kW h energy saving) per day in winter in comparison to a commonly suggested method of intensive and brief airing. At the same time, it could ensure indoor CO2 concentration to keep within the pre-set ranges (Pettenkofer limit: 1000 ppm) as well as low variations of indoor temperature (standard deviation (SD): 0.1°C)

Restricting the Maximum Number of Actions for Decision Support Under Uncertainty

Standard approaches for decision support are computing a maximum expected utility or solving a partially observable Markov decision process. To the best of our knowledge, in both approaches, external restrictions are not accounted for. However, restrictions to actions often exists, for example in the form of limited resources. We demonstrate that restrictions to actions can lead to a combinatorial explosion if performed on a ground level, making ground inference intractable. Therefore, we extend a formalism that solves a lifted maximum expected utility problem to handle restricted actions. To test its relevance, we apply the new formalism to enterprise architecture analysis